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ABSTRACT 



Empirical evaluations of learning with hypertext have shown 
contradictory results. Adaptive hypertext was introduced to solve some 
problems when learning with hypertext. This paper reports on two empirical 
studies comparing different forms of adaptive hypertext. In the first 
experiment, four treatments were realized by a combination of adaptive 
annotation and incremental linking of hypertext. Results showed significant 
improvements in knowledge tests for subjects learning with adaptive 
annotation and incremental learning. In the second study, different forms of 
adaptive annotation were integrated in the World Wide Web-based learning 
environment AST (Adaptive Statistics Tutor) . Results showed a significant 
interaction between previous knowledge of students and the adaptive 
annotation of the learning materials. Students who performed well in an 
introductory knowledge test seemed to work more intensively and successfully 
with a more supportive form of adaptive annotation, while students with less 
previous knowledge seemed to need more guidance and profit from a more 
restrictive form of adaptation. Three tables present data, and five figures 
illustrate the main pages of hypertext in the four experimental treatments 
and the interaction between the results of a preliminary knowledge test and 
the adaptive treatment. (Author/DLS) 
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Abstract: Empirical evaluations of learning with hypertext have shown contradictory results. Adaptive hypertext was 
introduced to solve some problems when learning with hypertext. In the following paper two empirical studies comparing 
different forms of adaptive hypertext are described and the results are discussed. In the first experiment four treatments where 
realized by a combination of adaptive annotation and incremental linking of hypertext. Results show significant better 
improvements in knowledge tests for subjects learning with adaptive annotation and incremental linking. In a second study 
different forms of adaptive annotation where integrated in the WWW-based learning environment AST. Results showed a 
significant interaction between previous knowledge of students and the adaptive annotation of the learning materials. Students 
who performed well in an introductory knowledge test seemed to work more intensively and successful with a more supportive 
form of adaptive annotation, while students with less previous knowledge seemed to need more guidance and profit from a 
more restrictive form of adaptation. 



1 Introduction 

Empirical studies have shown contradictory results about the efficiency and effectiveness of learning with 
hypertext as compared to linear textbooks. The benefit of learning with hypertext seems to be dependent on 
various factors like previous proficieny in the domain, the complexity of the domain or the learners motivation. 
Arguments for better learning with hypertext are the cognitive plausability and assumptions about the acquisition 
of meta-cognitive knowledge. However there are some problems with hypertext often mentioned in the literature. 
Students loose orientation or are overwhelmed by the mass of information presented to them [Conklin 1987; 
Conklin & Begeman 1988]. 

Adaptive hypertext was introduced as one possible solution to overcome these problems of learning with classical 
hypertext,. Adaptive methods in hypertext try to make an adequate presentation and selection of content for 
individual learners. However, beside the argumentation that adapted hypertext must be a better learning medium 
because information and presentation is tailored to the student, empirical studies on the effects of learning with 
adaptive hypertext are rarely found. Furthermore differential effects between various adaptive methods are not 
evaluated in experimental studies until now. The need for differentiating between adaptive methods becomes 
obvious when considering the consequences of these adaptive methods for the learner. For example, the method of 
incremental linking 1 hides hyperlinks that in some sense are not adequate for the learner. This restricts the 
learner’s navigational freedom and can have important drawbacks on learning. On the other hand adaptive 
annotation like introduced in ELM- ART [Weber & Specht 1997] gives adaptive navigation support to the learner 
without any restrictions on the content or navigational level. 

In this paper two empirical studies are presented which compare learning with hypertexts that were enhanced with 
adaptive annotation and incremental linking. The first study compares learning with adaptive hypertexts and a 
static hypertext in an experimental setting. The second study compares three forms of adaptive annotation in an 
WWW based learning environment on statistics: AST [Specht, Weber, Heitmeyer, & Schoch 1997]. 

1.1 Adaptive annotation as a learning aid 

In his review of adaptive methods in hypermedia Brusilovksy [1996] distinguishes two major groups of adaptive 
methods: methods of adaptive presentation and methods of adaptive navigation support. 

In this paper the focus will be on methods of adaptive navigation support, especially adaptive annotation and 
incremental linking. Methods of adaptive navigation support differ in their impact on learners freedom to navigate 



1 The main idea of incremental linking is to incrementally introduce hyperlinks in a document depending on the knowledge a 
student has already acquired. Implicitly this method restricts the navigational freedom of a learner through hiding hyperlinks 
to complex contents. 
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in a hyperspace. Furthermore, combinations of adaptive methods can restrict the users of a system in varying ways. 
The amount of learner control seems to.be a central variable when integrating adaptive methods in educational 
settings. In the literature there have been mentioned several arguments for and against learner control: 

Acquisition of metacognitive skills : Devoper und Quintin [1992] stress the importance of learner control for the 
acquisition of metacognitive skills. They argue that the amount of learner control should be dependent from the 
age, the domain knowledge or the complexity of learning materials. Greer and McCalla [1993] criticize a surplus 
of confidence in the abilites of learners for self controlled learning. 

Acceptance : Restricting learners in learning environments can be frustrating for them and therefore be rejected by 
the learners. The amount of control of a learner and the acceptance of a "teaching system" is dependent on the 
preskills and the knowledge state of a learner. Novices often want to have some guidance, while experts are just 
annoyed by restrictions. 

Additional mental load through learner control: Beside the freedom for learners learner control requires additional 
efforts. Decisions about where to navigate and what is best to be learned next have to be made by the learner. This 
adds meta-tasks of monitoring and self-evaluation to the the learners tasks of learning the domain. Hammond and 
Allison [1992] showed that the improvements of learning through learner control depend on the previous 
experience of a learner the subject matter. Secondary and metaanalysis of learner control show disappointing 
results: although motivation and engagement of learners seem to increase through learner control, results in 
performance tests are mostly better in adaptive and systemcontrolled learning environments. 

1.2 An experiment on adaptive annotation and incremental linking in hypertext 
1.2.1 Method 

In the following study learning with three different forms of adaptive hypertext and learning with a static hypertext 
were compared. The four experimental treatments where realized by a combination of the two adaptive methods of 
adaptive annotation and incremental linking. 

Adaptive annotation of hyperlinks supplies the user with additional information about the content behind a 
hyperlink. The annotation could be adapted to the individual user by taking into account different aspects of the 
learner and relations of the contents to be learned. In the following study information about the learner’s 
knowledge state and a didactic model of the domain concepts were used to adapt the presentation of coloured balls 
in front of each hyperlink. The knowledge state of a learner was represented in the learner model by simply storing 
if a concept was seen by the learner. The didactical model of the domain specified prerequisites (concepts) for 
each domain concept. Concepts which had been visited by the learner were annotated with a hook, concepts which 
the learner was not ready to learn (because of lacking prerequisites) were presented with a red ball, and 
recommended concepts were annotated with a green ball. 



Tablet: Experimental treatments from combining adaptivce annotation and incremental linking 





Adaptive annotation 


No annotation 


Incremental linking 


Treatment: IncAnno (Fig.l) 

• Restricted navigational freedom 

• Additional information through adaptive 

annotation 

• Transparency for the learner (All hyperlinks 

were visible from the beginning) 


Treatment: Inc (Fig. 2) 

• Restricted navigational freedom 

• Surprising introduction of hyperlinks 

• No transparency 


Static linking 


Treatment: Anno (Fig. 3) 

• No restriction for navigation 

• Additional information through adaptive 

annotation 


Treatment: Text (Fig.4) 
No adaptation 



Incremental linking can be considered as a special way of adaptive hiding. In contrast to adaptive annotation, 
adaptive hiding restricts the learners freedom to navigate in a hyperspace. While adaptive annotation just gives 
additional information adaptive hiding simply hides some information that is not adequate for the learner in the 
current situation. Based on the didactic model and the learner model concepts which the learner was not ready to 



read are hidden. In the following experiment the information that was not adequate for the learner was not directly 
hidden from the screen but there was no way to get to this information. Concepts the learner was not ready to visit 
(cause of missing prerequisites) where not presented as a link in the hypertext. 

From a combination of these two adaptive methods four different experimental treatments were realized, which are 
shown in [table 1]. In the experimental treatment IncAnno both adaptive methods were applied to present the 
hypertext. Consequently learners were restricted in their freedom to navigate the hypertext because of incremental 
linking. Learners had full transparency of the whole hyperspace because they could see where they need to go and 
where they will be allowed to hyperjump later on. [Figure 1] shows the hypertext in the treatment IncAnno. In the 
treatment Inc learners could not see where they will be allowed to jump later on, only concepts that were 
appropriate to their current knowledge state were presented as a hyperlink [Figure 2]. In the adaptive condition 
Anno all hyperlinks were presented from the beginning, learners were not restricted in their navigational freedom 
and had additional information about the adequacy of hyperlinks [Figure 3]. In the last experimental condition a 
static hypertext with no adaptation was presented to the learners [Figure 4]. 

Schwachsinn 

In der Geschichte der 3 Infektionskrankhei ten werden seit etwa 200 Jahren 
9 Prionenkrankhei ten beim ^ Mensch und beim ® Tier beschrieben. Ihr 
3 Erreger wie auch der G Obertragungsweg konnten aber lange Zeit nicht- 
identif i ziert werden, trotz umfassender Kenntnisse aber Symptomatik, 
zeitlichenTtolauf und Epidemiologic . ^ ^ _ 

Figure 1: The main page of the hypertext in the experimental treatment IncAnno 

Schwachsinn 

In der Geschichte der Infektionskrankhei ten werden seit etwa 200 Jahren 
Prionenkrankhei ten beim Mensch und beim Tier beschrieben. Ihr Erreger wie auch 
der Obertragungsweg konnten aber lange Zeit nicht, identifiziert werden, trotz 
umfassender Kenntnisse aber Symptomatik, zeitlichen Ablauf und Epidemiologie. 



Figure 2: The main page of the hypertext in the experimental treatment Inc . 

Schwachsinn 



In der Geschichte der ^ Infektionskrankhei ten , werden seit etwa 200 Jahren 
^ Prionenkrankhei ten beim Mensch und beim fr Tier beschrieben. Ihr fr Erre qer 
wie auch der ^ Obertra gungsweg konnten aber lange Zeit nicht identif iziert 
werden, trotz umfassender Kenntnisse aber Symptomatik, zeitlichen fib lauf und 
Epidemiologie. 



Figure 3: The main page of the hypertext in the experimental treatment Anno. 

Schwachsinn 

In der Geschichte der Infektionskrankhei ten werden seit etwa 200 Jahren 
Prionenkrankhei ten beim Mensch und beim Tier beschrieben. Ihr Erreger wie auch 
der Obertra aunasweq konnten aber lange Zeit nicht identif iziert werden, trotz 
umfassender Kenntnisse aber Symptomatik, zeitlichen Ablauf und Epidemiologie. 



Figure 4: The main page of the hypertext in the experimental treatment Text. 
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The underlying hyperspace consisted of 16 concepts about prionic diseases. 12 concepts could be reached directly 
from the main page, while 4 only could be visited via another concept. The only static link in all treatments was 
the possibility to jump back to the main page from each concept. 

1.2.2 The experiment 

All subjects had to answer a demographic questionaire and a knowledge test about prionic diseases. The 
knowledge test included 12 questions about central concepts of the curriculum with variing difficulty. Next a short 
introduction to using hypertext and information about the specific experimental treatment was given. When the 
subjects had visited all hypemodes the system automatically presented the concluding questionnaire. This included 
the knowledge test from the beginning and additional questions about the usability and helpfulness of the adaptive 
methods. As the main dependent variables the time to read all hypemodes and the number of correctly answered 
questions was measured. 

1.2.3 Results 

85 subjects completed the experiment. In the demographic questionaire there were no differences in the experience 
with computers and the WWW experience between the four groups. In all experimental conditions there was a 
significant improvement of correctly answered questions from the introductory to the knowledge test at the end 
(F(l,81)=308 ; p*0.01). In the introductory knowledge test the treatment group Text had the best results, in the 
knowledge test at the end the group IncAnno showed the best results. For browsing the hypertext the group 
IncAnno needed less time than all other groups, subjects in the condition Text needed more time than all others. 
The means of the two knowledge tests and the time to browse the hypertext are shown in [table 2]. 



Table 2: The means of the four groups f or the knowledge tests and the time to browse the hypertext. 





Treatment 


Anno 


Inc 


IncAnno 


Text 


Questionaire before Learning 


4.67 


5.22 


4.88 


5.53 


Questionaire after Learning 


10.45 


10.0 


11.33 


9.94 


Mean time of browsing (in sec.) 


692 


765 


618 


893 



To be sure that the experimental groups didn’t differ in their previous knowledge about the domain a variance 
analysis was computed on the data of the introductory knowledge test, which showed no sigificant differences 
(F(3,81)=0.37 ; p*0.05). 

A variance analysis comparing the treatments for the second knowledge test showed a hardly not significant main 
effect for the adaptive annotation of hyperlinks (F( 1 ,8 1 )=3 .9 1 ; p=0.052) and no significant effect for the 
incremental linking (F(l,81)=2.410 ; p*0.05). Significant effects for both adaptive annotation (F( 1 ,8 1)=1 3.17 ; 
p*0.05) and incremental linking (F(l,81)=4.49 ; p*0.05) could be shown for the time to browse the hypertext. 
Comparing only the two experimental groups which had both adaptive methods ( IncAnno ) and which had no 
adaptivity (Text) showed a significant effect on the correctly answered questions (t(39)=2.38; p*0.05) and the time 
needed (t(39)=-4.23; p*0.05) to browse the whole hyperspace. 

1.2.4 Discussion 

The results show better improvements in a knowledge test when learning with adaptive hypertext than learning 
with static hypertext. Learners which were restricted through incremental linking needed less time to read a 
hypertext and did not report problems with restrictions. By a combination of adaptive annotation and incremental 
linking learners got information about the whole hyperspace and the concepts that can be learned when progressing 
through the curriculum. This kind of transparency led to significantly better performance in a concluding 
knowledge test than learners in the static hypertext condition. 

From our point of view the differences between the treatment Anno and IncAnno are interesting. Allthough there 
was the same navigational support available for learners in the two treatments, there were no restrictions to 
navigational freedom in the treatment Anno. Learners were not prevented from jumping to complex concepts 
before reading about the prerequisite concepts. On the one hand one may argue that this would evoke feelings of 
loss of control (which were not reported in a questionaire). On the other hand the restrictions lead to a more 
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effective and efficient learning of the curriculum in the experiment. Of course this difference could be dependent 
on the complexitiy of the curriculum tought. We see two main conclusions from the results of this study: 

1. The effects of adaptive methods in learning environments should be investigated in isolation and in view of 
synergetic effects between the methods. 

2. The main criticism on integrating adaptive methods in educational hypertext are dependent on various learner 
features, the setting and the curriculum. Following a more instructional approach could be appropriate for learning 
small curriculums in a effective and efficient way. A more constructivist approach could be appropriate for 
complex curriculi and domain experienced learners. 

1.3 A field study on adaptive annotation in a learning environment 

In the second study different forms of adaptive annotation were integrated into the WWW Courseware "Adaptive 
Statistics Tutor" (AST). AST is an adaptive learning environment for learning introductory statistics on the WWW. 
Learners can read texts, look at examples, play around with concepts by manipulating interactive java applets or 
html-forms, or work on tasks and tests. Students can start from any point in the curriculum. When a learner selects 
a new section the system checks if all prerequisites for that section are already covered by the student. If a student 
lacks some knowledge for working on that section, the system presents some tests for the lacking knowledge. 
Depending on the result of the tests the system infers that the learner already knows about the prerequisites or 
recommends some hyperlinks. 

1.3.1 Method 

Similar to the experimental conditions in the first study three adaptive treatments were realized in the AST 
learning environment. The three treatments resulted from a combination of adaptive annotation and adaptive 
hiding. 

7. ANNOTATION Annotation of Hyperlinks: With each hyperlink a colored ball was presented, which gave some 
information about the concept behind the hyperlink. The color of the balls was adapted to the knowledge state of a 
student. Green balls classified the corresponding link as a recommendation, orange balls were presented when all 
prerequisites to this concept were learned and red balls meant that the hyperlink leeds to a hypemode with 
prerequisites that were not yet learned by the student. 

2. HIDE Annotation of Hyperlinks and hiding of "red" hyperlinks : In this treatment adaptive annotation of 
hyperlinks was realized like in treatment 1. except that hyperlinks which lead to "not ready to be learned" 
hyperlinks were hidden. When a student had mastered all prerequisites of a concept than the hyperlink to this 
concept was made visible and presented with a orange ball. The annotation of hyperlinks with green balls was 
computed by the system taking into account the knowledge state of a student, the learning material that were 
already seen by the student and a didactic model for sequencing concepts and learning materials. 

3. STATIC Annotation of learned and not learned concepts: In the third treatment all annotations were done with 
white balls and hooks, so learners only got information about what concepts they already had learned (hook) and 
what concepts they needed to work on (white ball). 

1.3.2 A field study 

In the field study the courseware was accessible to students of the University of Trier. Before working with AST 
students had to fill out a demographic questionnaire and work on a knowledge test about the statistics curriculum. 
The curriculum contained 23 concepts in 8 sections about descriptive statistics. With each section and concept 5 to 
15 tests were stored. When learners mastered a certain amount of tests correctly the system assumed that the 
students had learned that concept. The students were allowed to work with AST as long and as much as they 
wanted and the system was able to keep the user model over multiple sessions. 

1.3.3 Results 

In a period of three months 180 subjects worked with AST. In the following study only 67 subjects are taken into 
account because they did more than 20 requests to the system. The number of requests and the requested type of 
learning material was dependent on the adaptive treatment. 



Table 3 : The mean number of requests splitted by type of learning material and the experimental treatment 





Introduction 


Text 


Test 


Summary 


ANNOTATE 


1.04 


1.0 


3.70 


.64 
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Mean count of 
URL requests 




Figure 5: The interaction effect between the results 
in a preliminary knowledge test and the adaptive 
treatment 



HIDE 


1.13 


.59 


5.39 


.68 


STATIC 


1.01 


.62 


2.17 


.40 



Subjects in the ANNOTATION group requested significantly more text material (F(l,2)=6.1 1 ; p*0.05) than the 
other groups, while the subjects in the HIDE condition requested more tests (F(l,2)=5.77;p*0.05) than the other 
groups. The mean number of requests for the different learning materials are shown in [table 3]: 

The number of requests was not confounded with the preferences for different materials specified in the 
introductory questionaire. In the knowledge test before learning with AST there were no differences between the 
experimental groups. After introducing a post-hoc split between students with good results and with bad results in 
the introductory knowledge test an interaction effect on the number of requests could be clarified. Students with 
good results in a preliminary knowledge test worked more intensive with the system when they were in the 
ANNOTATE group. Vice versa, stu-dents with bad results in the introductory test worked better (more requests) in 
the HIDE group. The interaction effect between the results in a preliminary know-ledge test and the adaptive treat- 
ment is shown in [figure 5]. 

1.3.4 Discussion 

The different learning materials requested by subjects of the different treatments could be evoked by the nature of 
the adaptive treatments. This becomes obvious when looking at the HIDE treatment: Students could only get to 
more complex contents when working on the tests of the visible concepts. Similar to the first experiment the 
restriction of learners to certain subparts of the hyperspace can have different effects which are reflected in the 
interaction effect with the domain knowledge of learners. Learners who have good working knowledge in the 
domain seem to prefer more navigational freedom like in the ANNOTATE treatment. Learners who do not have 
much previous knowledge of the domain seem to prefer more directive guidance and work better with restrictions 
of navigational freedom. 

2 General discussion and further work 

Two experiments evalutating the effects of adaptive methods in two different settings were reported. In the first 
study a combination of adaptive annotation and incremental linking showed the best improvements of the 
knowledge acquisition process. In the field study reported second the shortcomings of empirical investigations in 
complex learning environments prohibited as clear results as in the experimental study. Most of the students did 
not complete a concluding knowledge test. This made it impossible to compare the knowledge improvements in 
the different experimental treatments. Nevertheless, results showed that the type of adaptive annotation can have 
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important impacts on the learning process and the style of learning. Different forms of annotation can have 

consequences on motivation and the acceptance of a learning environment. This is concistent with earlier results of 

Weber and Specht [1997]. In the future interactional and synergetic effects of different adaptive methods, learner 

features and characteristics of the educational setting have to be investigated in more detail. 
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